The Global Optimization Geometry of Low-Rank Matrix Optimization

This paper considers general rank-constrained optimization problems that minimize a objective function ${f}( {X})$ over the set of rectangular notation="LaTeX">${n}\times {m}$ matrices have rank at most r. To tackle constraint and also to reduce computational burden, we factorize notation="LaTeX">$ {X}$ into {U} {V} ^{\mathrm {T}}$ where {U}$ {V}$ are {r}$ notation="LaTeX">${m}\times matrices, respectively, then optimize small . We characterize global geometry nonconvex factored problem show corresponding satisfies robust strict saddle property as long original f restricted strong convexity smoothness properties, ensuring convergence many local search algorithms (such noisy gradient descent) in polynomial time for solving problem. provide comprehensive analysis matrix factorization aim find such approximates given {X}^\star $ Aside from property, has no spurious minima obeys not only exact-parameterization case notation="LaTeX">$\mathrm {rank}( {X}^\star) = , but over-parameterization < under-parameterization > These geometric properties imply number iterative converge solution with random initialization.